In an initial step of a decoder using a Viterbi algorithm (commonly used to decode a trellis code used in asymmetric digital subscriber line (ADSL) communications systems), the decoder needs to compute the Euclidean distance of a received point to four points on a two-dimensional (2-D) grid, wherein each of the four points belong to one of four different cosets and is the closest point in its respective coset to the received point. For a given number of bits represented by an ADSL tone, there is a corresponding constellation diagram. Since the number of bits is finite, the constellation diagram also contains a finite number of points.
A commonly used technique to simplify the distance computations (and other computations) is to assume that the received point lies on an infinite grid. For example, the use of an infinite grid can preclude the need to compute squared terms. The use of an infinite grid also places a limit on the possible size of a squared distance. The limit on the size of a squared distance may have an impact upon possible sizes of path metrics.
A disadvantage of the prior art is that the constellation diagrams have a finite number of points. Therefore, the assumption of using an infinite grid is incorrect. The use of an infinite grid can result in a measurable performance decrease relative to optimal decoding, especially when the number of bits in the constellation is small.